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(54) Real time dispatcher 

(57) The present invention provides a process 
scheduleror dispatcher for a multiprocessor system for 
real time applications. This embodiment of the present 
invention proposes a dispatcher model that maintains a 
dispatch queue for each processor and a separate glo- 
bal dispatch queue for unbound higher priority real time 
threads. A processor has its own queue and a dispatch- 
er. Each queue has a separate schedule lock associated 
with it to protect scheduling operations. A processor's 
dispatcher selects a thread for execution from one of the 
queues in the system as a candidate thread to execute. 



When a candidate thread is selected for execution, the 
processor proceeds to verify against threads in the glo- 
bal real time queue and the processor's own dispatch 
queue to select a highest priority runnable thread in the 
system. Thus, the present invention allows the dispatch- 
er to prevent race conditions and minimize lock conten- 
tion while assuring that high-priority threads are dis- 
patched as quickly as possible. The present invention 
is implemented by a synchronization between the oper- 
ations of dispatching a thread and making a thread run- 
nable. 



CM 
< 
O 

o 

00 

o 

o. 
us 



561 



High Priority 
Real Time 
Queue 



Shared. 
Memory 



Processor 



Dispatch Queue I 
1 J 



513 



Processor 

r 1 



515 



Dispatch Queue 
2 1 



5 17^ 



Processor 
N 



521 



Dispatch Queue 
N 



Printed by Jouvo. 75001 PARIS (FR) 



1 



EP 0 806 730 A2 



2 



Description 

This invention relates to a method and apparatus 
for efficient process scheduling for a multiprocessor sys- 
tem in a real time environment. 

The multiprocessor system (MP) is characterized 
by the presence of multiple CPUs or processors which 
cooperate on common or shared computational tasks. 
By having multiple processors in a single computer sys- 
tem, one can obtain performance enhancement by al- 
lowing many processors to share the computational 
load or by allowing many smaller tasks to be performed 
in parallel in separate processors. A multiprocessor sys- 
tem can also improve the system reliability since the fail- 
ure of one processor will not necessarily lead to a crash 
of the entire system. 

The introduction of multiple processors, however, 
complicates the resource management and allocation 
problem since two or more processes execute simulta- 
neously in the kernel on separate processors. Thus, a 
scheduler module of an operating system needs to de- 
cide which processor should execute a process and in 
its general form, schedule a set of processes on a set 
of processors with arbitrary characteristics in order to 
optimize some objective function. This involves the se- 
lection of a process for execution from a set of process- 
es. 

The basic abstractions in an operating system in- 
clude "tasks" and "threads." A task is a unit of resource 
management and a thread is a single flow-of-control. 
Each thread possesses a register state and a stack. The 
system associates with each thread extra state informa- 
tion relating to its schedulability. These include the 
thread dispatch priority and processor affinity that de- 
termines on which processors a thread can execute. 

Figure 1 (A) shows a prior art single queue dispatch- 
er for a multiprocessor system wherein a dispatcher us- 
es an array 1 01 of dispatch queues, indexed by dispatch 
priority. In Figure 1 (A), when a thread is made runnable, 
it is placed on a dispatch queue, typically at the end, 
corresponding to its dispatch priority. When a processor 
switches to a new thread, it always selects the thread at 
the beginning of the highest priority non-empty dispatch 
queue. Threads may not change dispatch priority while 
on a dispatch queue; the thread must be first removed, 
its dispatch priority adjusted, and then the thread may 
be placed on a different dispatch queue. 

Figure 1 (B) illustrates a prior art single queue dis- 
patcher where a thread that needs to wait on a synchro- 
nization object is placed on a sleep queue 102 associ- 
ated with the synchronization object. The sleep queue 
is maintained in dispatch priority order, so that when the 
synchronization object is released, the highest priority 
thread waiting for the object is at the head of the sleep 
queue. 

The prior art system shown in Figures 1 (A) and 1 
(B) uses a single spin schedule lock to protect all sched- 
uling operations. Each scheduler may obtain or release 



the lock. When the schedule lock is currently held by a 
processor, other processors that need the schedule lock 
"spin" on the lock, waiting for access. In particular, 
whenever the release of a synchronization object makes 
s some thread runnable, the schedule lock is held while 
placing the thread on the dispatch queue. To prevent 
interference and delays from interrupt routines, the hold- 
er of the schedule lock runs at an elevated interrupt lev- 
el. 

Thus, in the prior art system shown in Figures 1 (A) 
and 1 (B), the contention for a single schedule lock can 
cause processors to wait in line to make a scheduling 
decision, resulting in some idle processors. As a result, 
not only the whole purpose of using multiple processors 
to achieve faster job processing is defeated, but valua- 
ble computer resources go unused. 

This situation can be somewhat ameliorated by in- 
troducing a multiple dispatch queue system wherein 
each processor is allowed to have its own dispatch 
queue and its own schedule lock. Such a multi-queue 
multi-lock system is shown in Figure 2(A) wherein each 
processor maintains its own dispatcher queue. For ex- 
ample, there is a separate dispatch queue associated 
with each of processors 1 , 2, N in Figure 2(A). 

In the system of Figure 2(A), a processor ready to 
take a new thread starts searching for a runnable thread 
in its own dispatch queue. When its own queue is not 
empty, the processor takes the highest priority runnable 
thread available off its own queue. However, when its 
own queue is empty, the processor starts searching oth- 
er dispatch queues for available threads. Still referring 
to Figure 2(A), if there is an available runnable thread 
at any other dispatch queue, unless the thread is 
marked as only runnable on a particular processor, the 
processor can take the thread from the other dispatch 
queue and execute the thread. However, prior art dis- 
patchers such as shown in Figures 1 and 2 are not suit- 
able for real time applications. 

A real time computing system is designed to deliver 
a required level of service or processing within a bound- 
ed time. Real time computing systems find applications 
in the areas of virtual reality, factory automation, robot- 
ics, teleconferencing, and multimedia broadcast sys- 
tem. These applications are typically "mixed-mode," 
that is, they are partitionable into schedulable entities, 
some of which require real time response. To achieve a 
bounded time response, time-critical applications re- 
quire control over their scheduling behavior. 

When a multiprocessor system is utilized to support 
real time applications, the scheduling of processes be- 
comes even more complicated and consequently a real 
time operating system must be able to provide some real 
time capability to time critical tasks. Thus, a real time 
system must be capable of providing immediate re- 
sponse to specific external events and scheduling par- 
ticular processes to run within a specified time limit after 
occurrence of an event. A real time system also needs 
to guarantee that the operating system can schedule a 
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particular process within a fixed time limit. 

To service real time high priority threads as rapidly 
as possible, the system in Figure 2(A) can be improved 
to have a separate real time queue as shown in Figure 
2(B). In Figure 2(B), an additional superqueue is added 
at a higher level of multiple dispatcher queues to provide 
system-wide visibility of runnable real time threads. The 
superqueue is a queue of dispatch queues containing 
those processor queues that hold unbound real time 
threads with priority higher than some predetermined 
threshold priority level. Thus the superqueue lists those 
processors whose highest priority thread has sufficient 
priority to be considered a real time thread. 

Referring to Figure 2(B), a processor first searches 
for runnable threads in the superqueue. If there is a non- 
empty high priority queue in the superqueue, the proc- 
essor takes the highest priority runnable thread off that 
queue. If the superqueue is empty, the processor 
checks its own queue. If its own queue is still empty, 
then the processor goes on to other dispatcher queues 
for runnable threads. If there is any non-empty dispatch- 
er queue in the system, the processor takes the highest 
priority runnable thread off that queue and starts exe- 
cuting the thread. 

Real time threads would be dispatched by examin- 
ing the superqueue. Thus, when a processor is selecting 
a thread to run, it first examines the superqueue and 
then its own queue. 

Although the above solution outlined with respect 
to Figure 2(B) represents an improvement over prior art 
and offers a straightforward solution to real time multi- 
processor scheduling, it presents a critical race problem 
in scheduling real time threads. For example, a race 
condition can be created under the following scenario: 
suppose processors 1 and 2 are notified that there are 
two real time threads that are runnable in the super- 
queue. Processors 1 and 2 proceed to check the super- 
queue to find which processor queues have the two 
threads. The two real time threads are in, for example, 
processor 3 and processor 4's dispatch queues with 
the'real time thread in processor 3's queue having high- 
er priority than the one in processor 4's queue. 

After finding from the superqueue that the highest 
real time thread is in processor 3's dispatch queue, both 
processors 1 and 2 attempt to access, and take the real 
time thread off, processor 3's queue. However, proces- 
sor 1 accesses the processor 3's queue first and takes 
the real time thread. Then processor 2, still looking for 
a thread in processor 3's queue and not knowing that 
the highest priority real time thread has just been taken 
by processor 1 , attempts to take the next highest priority 
thread off processor 3's queue. The next highest runna- 
ble thread in processor 3's queue, however, may be a 
very low priority one. As a result, the real time thread in 
processor 4's queue is left in the queue waiting to be 
scheduled until some processor becomes available to 
service it. 

In the above scenario, which is an example of many 



possible disadvantages due to imprecise synchroniza- 
tion, the lack of communication between processors 1 
and 2 leads to a race condition, which results in higher 
priority real time thread waiting in a queue while a lower 
5 priority thread is being serviced. Imprecise system syn- 
chronization, thus, can cause missed deadlines for real 
time threads even under low levels of processor utiliza- 
tion. 

Thus, prior art dispatchers, fraught with race condi- 
tions and errors in the implementation of the dispatcher, 
can result in non-real time behavior for real time (RT) 
threads and in some cases can hang up the system. 

Thus, presently, a variety of apparatus and methods 
are employed to provide a dispatcher for a multiproces- 
sor system. However, they are not suitable for real time 
applications, do not deliver real time service for many 
applications, and are fraught with race conditions. Ac- 
cordingly, it would be advantageous to provide a dis- 
patcher that can efficiently support both time sharing 
(TS) and real time (RT) scheduling policies for a multi- 
processor system. 

The present invention provides a process scheduler 
or dispatcher for a multiprocessor system for real time 
applications. In the embodiments of the present inven- 
tion, each processor has its own queue and a dispatcher 
so that the system can maintain a dispatch queue for 
each processor and a separate global dispatch queue 
for unbound higher priority real time threads. Each 
queue has a separate schedule lock associated with it 
to protect scheduling operations. A processor is allowed 
to place a new thread on the global high priority real time 
queue, the processor's own queue, or any other proc- 
essor's queue. 

A processor's dispatcher can select a thread for ex- 
ecution from the global real time queue, a processor's 
own queue, or another processor's queue as a candi- 
date thread to execute, based upon priority variables as- 
sociated with each queue. The examination of the pri- 
orities on queues for thread selection does not require 
any schedule locks and miscommunication is prevented 
by using a suitable synchronization algorithm. When a 
candidate thread is selected for execution, the proces- 
sor notifies other processors of its selection and pro- 
ceeds to verify against threads in the global real time 
queue and the processor's own dispatch queue to select 
the highest priority runnable thread in the system. 

Thus, the present invention allows the dispatcher to 
prevent race conditions and minimize lock contention 
while assuring that high-priority threads are dispatched 
as quickly as possible. The preferred embodiment of the 
present invention is implemented by a synchronization 
between the operations of dispatching a thread and 
making a thread runnable. 

The invention will now be described by way of ex- 
ample with reference to the following drawings 

Figure 1 (A) shows a prior art single queue dispatch- 
er system for a multiprocessor environment wherein a 
dispatcher uses an array of dispatch queues, indexed 
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by dispatch priority. 

Figure 1 (B) shows a prior art single queue dispatch- 
er system wherein blocked threads are waiting on a syn- 
chronization object. 

Figure 2(A) shows a multi-queue multi-lock system. 

Figure 2(B) shows a multi-queue multi-lock system 
with an additional superqueue. 

Figure 3 illustrates a general purpose computer 
suitable for implementing an embodiment of the inven- 
tion. 

Figure 4(A) shows a preferred embodiment of mul- 
tiple dispatcher queue system according to the present 
invention. 

Figure 4(B) shows a dispatch queue structure in 
greater detail using dispatch queue for a processor of 
Figure 4(A). 

Figure 5 shows a preferred configuration of the mul- 
tiprocessor system according to the present invention. 

Figure 6 shows a flowchart describing a dispatcher 
scheduling a thread to run on a processor. 

Figure 7 shows a flowchart describing in greater de- 
tail the thread selection process of step 601 of Figure 6. 

Figure 8 illustrates a global priority mapping strate- 
gy- 
Figure 9 shows a flowchart describing a processor 
making a thread runnable. 

Figure 10 illustrates a flowchart describing in great- 
er detail the processor selection step 905 of Figure 9. 

The present invention is a multiprocessor schedul- 
ing system described with respect to real time applica- 
tions. In the following description, numerous specific de- 
tails are set forth to provide a more thorough description 
of the present invention. It will be apparent, however, to 
one skilled in the art, that the present invention may be 
practiced without these specific details. In other instanc- 
es, well known features have not been described in de- 
tail so as not to obscure the present invention. 

In a multiprocessor system, there are basically two 
resource allocation decisions that are made. One is 
where to locate code and data in physical memory and 
the other is on which processor to execute each process 
- an assignment decision or processor management. 
These decisions are challenging in a multiprocessor 
system and optimization of the processor management 
is important in a multiprocessor system. 

Assignment decision or job and processor schedul- 
ing includes multilevel scheduling policies: high-level 
scheduling or job scheduling that determines which jobs 
shall be allowed to compete actively for system resourc- 
es; intermediate-level scheduling that determines which 
processes shall be allowed to compete for system re- 
sources; and low-level scheduling performed by a dis- 
patcher. A dispatcher determines which ready process 
will be assigned to which processor when it next be- 
comes runnable. 

Afully preemptive scheduling scheme is used in the 
preferred embodiment of the present invention, in which 
a processor can be taken away from a process that it is 



currently running to let other higher priority process 
come in for execution. Although preempt ton involves 
overhead due to context switching, it is useful in systems 
in which high priority processes require rapid attention. 

5 In real time systems, for example, missing a hard dead- 
line can bring fatal consequences and thus, preemptive 
scheduling is necessary in guaranteeing acceptable 
service times. 

In the present invention, a process scheduler or dis- 

10 patcher performs its function each time a running proc- 
ess is blocked or preempted. Its purpose is to select the 
next running process from the set of ready queues. The 
dispatcher resides in the operating system kernel, mon- 
itoring the ready queues and handling requests to load 

is applications. This involves creating all the tasks and 
threads of the application, reserving memory, and load- 
ing the code and data into the memory. All the resources 
are reserved before an application is considered suc- 
cessfully loaded and ready to run. Thus, a dispatcher 

20 should be fairly efficient to minimize operating system 
overhead. 

In a multiprocessor system, real time priority sched- 
uling must meet one constraint: for each processor, 
there is no higher priority thread on any queue than can 
25 be run on that processor. From throughput point of view, 
it is best that each processor has its own dispatch 
queue, thus minimizing the lock contention between 
processors. 

This embodiment of the present invention, there- 
to fore, proposes a dispatcher model that maintains a sep- 
arate global dispatch queue for unbound higher priority 
real time threads in addition to processors' own dispatch 
queues. Further, instead of guaranteeing that the dis- 
patcher always makes correct decisions, this embodi- 
es ment of the present invention uses a select and verify 
scheme. Both of these features allow this embodiment 
of the present invention to minimize lock contention 
while assuring that high-priority threads are dispatched 
as quickly as possible. This is implemented by a syn- 
40 chronizatton between the operations of dispatching a 
thread and making a thread runnable. The dispatcher 
model of this embodiment of the present invention is im- 
plemented in SunOS Solaris 2.5. 

The present invention can be implemented on a 
45 general purpose computer such as illustrated in Figure 
3. A keyboard 310 and mouse 311 are coupled to a bi- 
directional system bus 318. The keyboard and mouse 
are for introducing user input to the computer system 
and communicating that user input to CPU 313. The 
so computer system of Figure 3 also includes a video mem- 
ory 314, main memory 315 and mass storage 312, all 
coupled to bi-directional system bus 31 8 along with key- 
board 31 0, mouse 31 1 and CPU 31 3. The mass storage 
312 may include both fixed and removable media, such 
ss as magnetic, optical or magnetic optical storage sys- 
tems or any other available mass storage technology. 
Bus 318 may contain, for example, 32 address lines for 
addressing video memory 314 or main memory 315. 
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The system bus 31 8 also includes, for example, a 32-bit 
DATA bus fortransferring DATA between and amongthe 
components, such as CPU 31 3, main memory 31 5, vid- 
eo memory 314 and mass storage 312. Alternatively, 
multiplex DATA/address lines may be used instead of s 
separate DATA and address lines. 

Main memory 31 5 is comprised of dynamic random 
access memory (DRAM). Video memory 314 is a dual- 
ported video random access memory. One port of the 
video memory 31 4 is coupled to video amplifier 31 6. The 
video amplifier 316 is used to drive the cathode ray tube 
(CRT) raster monitor 317. Video amplifier 316 is well 
known in the art and may be implemented by any suit- 
able means. This circuitry converts pixel DATA stored in 
video memory 314 to a raster signal suitable for use by 
monitor 317. Monitor 317 is a type of monitor suitable 
for displaying graphic images. 

The computer system described above is for pur- 
poses of example only. The present invention may be 
implemented in any type of computer system or pro- 
gramming or processing environment. 

Figure 4(A) shows a multiple dispatcher queue sys- 
tem according to a preferred embodiment of the present 
invention. Referring to Figure 4(A), the multiple dis- 
patcher queue system comprises separate dispatcher 
queues 401, 402 403 for processors 1,2,..., N and in 
addition, a global high priority real time queue 404, 
which is used to hold high priority real time threads. 
Each dispatch queue has its own scheduling lock asso- 
ciated with it to protect all scheduling operations so that 
any processor attempting to dispatch a thread from a 
queue needs to acquire a lock for that queue before tak- 
ing the thread off the queue. Thus, lock contention is 
reduced in the present invention with multiple schedule 
locks for multiple dispatch queues in contrast to prior art 
schedulers wherein a single schedule lock is used for a 
single dispatch queue for all processors. 

In the preferred embodiment of Figure 4(A), a dis- 
patcher uses an array of dispatch queues, indexed by 
dispatch priority. Also, when a thread is made runnable, 
it is placed on a dispatch queue, typically at the end, 
corresponding to its dispatch priority. However, other 
schemes can be used for queueing instead of FIFO 
thread queueing. For example, LIFO (Last In First Out), 
SJF (Shortest Job First), SRT (Shortest Remaining 
Time) or other sophisticated queueing mechanism can 
be used depending on real time application scheduling 
requirements. 

Figure 4(B) shows a dispatch queue structure in 
greater detail using dispatch queue for processor 1 of 
Figure 4(A). In Figure 4(B), threads 1 , 2, and 3 of some 
priority value are queued in the dispatch queue. Thread 
1 is on top of the queue and, if a FIFO system is applied, 
would be the first thread in that priority category to be 
dispatched from the queue for execution. Threads in 
Figure 4(B) also have processor affinity characteristics 
associated with them. 

A processor affinity is used to determine on which 



processors a thread can execute. Most of threads are 
executable on all processors and are accordingly la- 
beled so. However, there are threads that are only exe- 
cutable on a particular processor and no other proces- 
sors. This thread then is given a processor affinity for 
that particular designated processor. When a thread has 
an affinity for a particular processor, it cannot be stolen 
by, or migrate to, other processors. Thread 2 in Figure 
4(B), for example, has a processor affinity for processor 
1 and is only executable on processor 1 . Thus a thread 
that is only runnable on one processor, i.e. a bound 
thread, can only appear on its bound processor's dis- 
patch queue and will only be dispatched by that proces- 
sor. 

For a processor to be able to steal a thread from 
other processors, the thread must be runnable on the 
stealing processor. For example, referring to Figure 4 
(B), threads 1 and 3 can be stolen, taken off the queue, 
and executed by other processors than processor 1 . 

Figure 5 shows a configuration of the mutiprocessor 
system according to a preferred embodiment of the 
present invention. Each processor 1 , 2,..., N has its own 
scheduler and dispatch queue as shown in Figure 5. For 
example, processor 1 has scheduler 505 and dispatch 
queue 509 coupled to it through a bus. Schedulers or 
dispatchers determine when and which threads are to 
be dispatched for execution on the system processors. 
Schedulers 1 , 2, .„, N and therefore processors 1,2,..., 
N are coupled via a bus to global high priority real time 
queue 501 and shared memory 503. Threads can inter- 
act using synchronization objects that are shared by all 
processors. 

Associated with each processor in Figure 5 are a 
set of scheduling variables used for various purposes. 
For example, variables cpu_runrun and cpu_kprunrun 
record requests for preemption of the current thread and 
are used to make scheduling decisions. Other variables 
include cpu_dispthread and cpu_thread. The variable 
cpu_thread refers to the thread currently executing on 
a processor. The variable cpu_dispthread is used for 
communication with other processors and records the 
identity of the thread last selected for dispatch on the 
processor. 

The variable cpu-runrun or cpu_kprunrun is set for 
a processor by other processors when they put a new 
thread in the processor's queue and find that the proc- 
essor is running a thread of lower priority than the new 
thread just put on the processor's queue. The variable 
cpu_kprunrun has higher priority than the variable 
cpu_runrun. When variable cpu-runrun or cpu_kprunrun 
is set, the processor needs to make a scheduling deci- 
sion to preempt the current thread and service a new 
higher priority thread. 

Any processor in Figure 5 is capable of putting a 
thread on any of the queues in the system for execution. 
For example, processor 1 can put a new thread on its 
own queue, the global high priority real time queue, or 
other processors' queues unless the new thread has a 



15 



20 



25 



30 



35 



40 



45 



50 



5 



9 



EP0 806 730 A2 



10 



particular processor affinity characteristic. 

Although Figure 5 shows multiprocessors connect- 
ed via a single bus, other alternate architectures are cer- 
tainly possible for an embodiment of the present inven- 
tion. For example, multibus multiprocessor organization 5 
or crossbar switch system can be used for interconnec- 
tion network to enhance the multiprocessor system per- 
formance. 

To dispatch or schedule a thread on a processor, 
the processor needs to find a thread to run. Figure 6 
shows a flowchart describing a dispatcher scheduling a 
thread to run on a processor using a select and verify 
scheme. 

Referring to Figure 6, a processor ready for a next 
thread starts to select a thread to run by checking in the 
high priority real time queue to see if it has any entries 
at step 601 . Figure 7 shows a flowchart describing in 
greater detail the thread selection process of step 601 . 

The examination of the priorities on queues involve 
checking local variable associated with each processor 
and does not require any locks. For example, priority 
variable disp_maxrunpri can be used to indicate maxi- 
mum priority level on a queue. Then variable 
disp_maxrunpri can be checked on both the processor 
dispatch queue and the real time queue using some suit- 
able synchronization algorithm such as Dekker"s algo- 
rithm to prevent miscommunication. However, any other 
suitable synchronization algorithm can be used in alter- 
nate embodiments of the present invention. Since the 
priority variables to be examined are atomic variables 
that are maintained in each dispatch queue, any sched- 
uling errors caused by selecting a wrong queue will be 
caught in the a verification step. However, a schedule 
lock is required to take a thread from a selected queue. 

Referring now to Figure 7, if the real time queue has 
a higher priority thread than its own dispatch queue at 
decision block 701 , the processor dispatcher acquires 
a lockfor the real time queue and takes a highest priority 
thread at step 702 from the real time queue such as 
queue 404 of Figure 4 and proceeds to execute the 
thread. 

A processor that becomes available to take a next 
thread first looks at the real time queue before checking 
their own queues, thereby servicing threads in the high- 
er priority real time queue ahead of threads in their own 
queues to provide the fastest possible service to real 
time threads. If, at decision block 701 , it is determined 
that the realtime queue has no higher threads available, 
the processor proceeds to step 703 to check its own dis- 
patch queue for a runnable thread. If its own queue is 
not empty, then the processor proceeds to step 704 to 
acquire a lock for its own queue and take a highest pri- 
ority thread off its own queue. 

If the processor's own dispatch queue is empty, 
then the processor proceeds to step 705 where it checks 
other processors' dispatcher queues to find a runnable 
thread. If there is a runnable thread with no particular 
processor affinity in any other dispatch queue, the proc- 



essor acquires a scheduling lock for that dispatch queue 
and proceeds to steal the thread from the other proces- 
sor at step 706. 

If the processor cannot find any non-empty dispatch 
queues or runnable threads in them at decision block 
705, the processor proceeds to step 707 to choose an 
idle thread, thereby completing the thread selection 
process. An idle thread is a special thread that has a 
tower priority than any dispatch priority and never ap- 
pears in the dispatch queue. An idle thread is selected 
for execution whenever no other thread is runnable, and 
switches whenever another thread becomes runnable. 

Thus, a thread can be selected for execution by a 
processor from any one of the high priority queue, its 
own dispatch queue, and other processors' dispatch 
queues in the system. However, by first checking the 
high priority real time queue such as 404 before check- 
ing its own queue or other dispatch queues, a processor 
gives higher priority to the real time queue than other 
dispatch queues. This global priority mapping is illustrat- 
ed in Figure 8. 

As shown in Figure 8, the time-sharing threads are 
given the lowest priority in job scheduling or dispatching. 
The time-sharing threads are supported by a time slicing 
technique and scheduled dynamically with a few hun- 
dred milliseconds per time slice. The time-sharing 
scheduler switches the context in a round-robin fashion 
often enough to give every thread an equal opportunity 
to run. The system threads having higher priority than 
the time-sharing threads comprise special system 
threads and interrupt threads. Interrupt threads are al- 
ways given the highest priority in the system. 

Real time threads are situated between interrupt 
threads and the system threads in Figure 8. Real time 
threads are scheduled strictly on the basis of their pri- 
ority and the time quantum associated with them. A time 
quantum or time slice is assigned to a thread to limit the 
amount of processor time allowed to run the thread. If a 
thread does not complete before its time quantum ex- 
pires, the thread is preempted and the next waiting high- 
est priority thread is dispatched to execute. For exam- 
ple, a real time thread with infinite time quantum runs 
until it terminates, blocks, or is preempted. 

The criteria for determining what priority level can 
be used as dividing lines for different priority regions in 
Figure 8, and therefore what priority thread qualifies to 
be in, for example, the global real time queue, can be 
set by considering a number of factors. For example, 
threads in the real time queue are optimized for respon- 
siveness while thread not in the real time queue are op- 
timized for throughput, maximizing the number of in- 
structions executed per unit time. 

Referring back to Figure 6, after selecting a candi- 
date thread to run at step 601, the processor proceeds 
to step 602 where it rel'eases the schedule lock it ac- 
quired in step 601 after broadcasting to other proces- 
sors a tentative notification that it will be running the se- 
lected candidate thread by setting local processor Vari- 
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ables, for example by setting variable cfu_dispthread to 
indicate the most updated thread priority. 

At step 603, the processor clears all notifications 
that this processor should reschedule by, for example, 
clearing cpu_runrun and cpu_kprunrun variables. How- 
ever, the processor still can not confirm that the selected 
candidate thread is the highest priority runnable thread 
it can take since there is a possibility that a new higher 
priority thread could have been placed in the global real 
time queue or in its own queue by some other proces- 
sors while the processor was in the thread selection 
process. This can create the race conditions as dis- 
cussed earlier in the background unless the scheduler 
can verify that the selected candidate thread is the best 
choice. 

Thus, at step 604, a verification is made as to 
whether the selected thread is a best possible selection. 
This requires going back to the high priority real time 
queue and its own queue, and checking them again to 
see if there is a higher priority thread newly placed in 
the high priority real time queue or its own queue. If the 
selected candidate thread has higher priority than any 
other thread in either queue, then the processor contin- 
ues to step 605 to execute the selected candidate 
thread. 

If, however, a new higher priority thread is found in 
either queue, the selecting processor puts the selected 
candidate thread back on some queue, probably on the 
queue the thread was taken off, depending on a queue 
placement algorithm, and goes back to step 601 to start 
the thread selection process anew. 

The select and verify scheme of this embodiment 
of the present invention, thus, guarantees that the final 
selected thread will indeed have the highest priority that 
the processor can select to execute. Also, the race prob- 
lems such as discussed in the background art are now 
prevented and higher priority threads in the system are 
guaranteed a real time service with minimum dispatch 
latency. 

Figure 9 shows a flowchart describing a processor 
making a thread runnable. At step 901, a processor is 
selected for the thread and at decision block 901 , a de- 
termination is made whether the thread is bound. If the 
thread is bound, the thread is put on the selected proc- 
essor's queue at step 902. If the thread is not bound, a 
determination is made at decision block 903 as to 
whether the thread has real time property. Real time 
threads are threads that have sufficient priority that re- 
sponsiveness of the system in dispatching the threads 
is more important than throughput. 

A predetermined threshold value can be used as a 
criterion to distinguish real time threads. For example, 
variable kppreemptpri is used in the preferred embodi- 
ment as the threshold value to determine real time 
threads. Variable kppreemptpri can be set to any apprb- 
priate value depending upon system applications. Thus, 
if the thread is unbound and its priority is above kp- 
preemptpri, the thread is put on the high priority real time 



queue at step 904. At step 905, a processor is selected 
to schedule the thread on before proceedin g to step 907. 
If, at step 903, it is determined that the thread has a pri- 
ority below kppreemptpri, then the processor proceeds 
s to step 906 to put the th read on the processor it last ran 
on and proceeds to step 907. 

Figure 10 illustrates the processor selection step 
905 of Figure 9 in greater detail. At step 1001, "best 
processor" is set to be the processor the thread last ran 
^0 on. Then each processor in the system determines if its 
current thread is of lower priority than the thread being 
inserted at step 1002. 

If its current thread is of higher priority, then it pro- 
ceeds to step 1005. Otherwise, a determination is made 
at step 1003 whether the processor's current thread is 
of lower priority than the thread on "best processor. - If 
the current thread is of lower priority, then at step 1004, 
"best processor" is set equal to current processor. At 
step 1005, "best processor" is selected as the target 
processor to schedule the thread being inserted. 

Turning back to Figure 9, after a target processor is 
selected to run the thread, a determination is made, at 
decision block 907, as to whether the thread has a high- 
er priority than the last thread the processor committed 
to. If the thread has a higher priority, the processor is 
notified at step 908 by setting some local variables, for 
example cpu_runrun and cpu_kprunrun variables. If the 
thread does not have a higher priority at step 907, the 
scheduling processor continues to step 909. 

It is understood that particular embodiments de- 
scribed herein should not limit the present invention 
thereby. This invention can be practiced in connection 
with any multiprocessor system that uses a preemptive 
priority scheduling system. 

Thus, a process scheduler or dispatcher for a mul- 
tiprocessor system suitable for real time applications 
has been described. 



40 Claims 

1 . A method for scheduling a thread in a multiproces- 
sor system based on a preemptive priority schedul- 
ing, said multiprocessor system comprising a plu- 
45 rality of processors, said method comprising: 

- selecting a thread as a candidate thread to ex- 
ecute from one of a plurality of local queues and 
a global queue, said plurality, of local dispatch 

50 queues for storing threads to be scheduled, 

each of said plurality of local dispatch queues 
coupled to one of said plurality of processors, 
said global queue for storing threads to be 
scheduled, said global queue accessible by 

55 each'of said plurality of processors; 

notifying processors of the candidate thread; 
checking whether a higher priority thread is 
available in its local queue and said global 
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queue; 

preempting the first selected thread and select- 
ing the higher priority thread as the candidate 
thread to execute when there is a higher priority 
thread; 

executing the candidate thread. 

2. The method of claim 1 wherein said step of select- 
ing a thread as a candidate thread to execute further 
comprises the step of selecting the highest priority 
thread from said global queue. 

3. The method of claim 2 wherein said step of select- 
ing a thread as a candidate thread to execute further 
comprises the step of selecting the highest priority 
thread from one of said plurality of local dispatch 
queues when there is no runnable thread in said 
global queue. 

4. The method of claim 3 wherein said step of select- 
ing a thread as a candidate thread to execute further 
comprises the step of selecting a thread from other 
processor's local dispatch queue when there is no 
runnable thread in said processor's local dispatch 
queue. 

5. The method of claim 4 wherein said step of select- 
ing a thread as a candidate thread to execute further 
comprises the step of selecting an idle thread when 
there is no runnable thread in said other processor's 
local dispatch queue. 

6. The method of claim 1 further comprising the step 
of: 

placing a thread on a processor's local dis- 
patch queue when the thread is bound to said proc- 
essor. 

7. The method of claim 6 further comprising the step 
of placing the thread on said global queue when the 
thread has real time property. 

8. The method of claim 7 further comprising the steps 

identifying a last processor that the thread ran 
on; 

placing the thread on the last processor's dis- 
patch queue when the thread does not have re- 
al time property. 

9. The method of claim 1 wherein said step of notifying 
processors of the candidate thread comprises: 

changing a memory register value, said reg- 
ister accessible by said plurality of processors. 

10. A multiprocessor scheduling system based on a 



preemptive priority scheduling, said multiprocessor 
scheduling system comprising: 

a plurality of processors; 
5 a plurality of schedulers, each of said plurality 

of processors coupled to one of said plurality of 
schedulers; 

a plurality of local dispatch queues, each of said 
plurality of processors coupled to one of said 
10 plurality of local dispatch queues; 

said plurality of schedulers coupled to a com- 
munication medium; 

a global dispatch queue coupled to said com- 
munication medium; 
15 a shared memory coupled to said communica- 

tion medium. 

11. The multiprocessor scheduling system of daim 11 
wherein said global dispatch queue stores real time 

20 threads. 

12. The multiprocessor scheduling system of claim 11 
wherein said shared memory comprises a register 
accessible by said plurality of schedulers. 

25 

13. An article of manufacture comprising: 



a computer usable medium having computer 
readable program code embodied therein for 

30 scheduling a thread in a multiprocessor system 

based on a preemptive priority scheduling, said 
multiprocessor system comprising a plurality of 
processors, the computer readable program 
code in said article of manufacture comprising: 

35 computer readable code configured to cause 

one of said plurality of processors to select a 
candidate thread to execute using a thread se- 
lect and verify method comprising the steps of: 

40 selecting a thread as a candidate thread to 

execute from a plurality of local dispatch 
queues and a global queue, said plurality 
of local dispatch queues for storing threads 
to be scheduled, each of said plurality of 

45 local dispatch queues coupled to one of 

said plurality of processors, said global 
queue for storing threads to be scheduled, 
said global queue accessible by each of 
said plurality of processors; 

50 notifying processors of the candidate 

thread; 

checking whether a higher priority thread 
is available in its local queue and said glo- 
bal queue; 

55 preempting the first selected thread and 

selecting the higher priority thread as the 
candidate thread to execute when there is 
a higher priority thread; 
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computer readable code configured to cause 
said one ol said plurality of processors to exe- 
cute the selected candidate thread. 

14. The article of manufacture of claim 1 3 wherein said 5 
computer readable code configured to cause one 
of said plurality of processors to select a candidate 
thread comprises computer readable code config- 
ured to cause one of said plurality of processors to 
select the highest priority thread from said global 
queue. 

15. The article of manufacture of claim 14 wherein said 
computer readable code configured to cause one 
of said plurality of processors to select a candidate 
thread comprises computer readable code config- 
ured to cause one of said plurality of processors to 
select the highest priority thread from one of said 
plurality of local dispatch queues when there is no 
runnable thread in said global queue . 

16. The article of manufacture of claim 15 wherein said 
computer readable code configured to cause one 
of said plurality of processors to select a candidate 
thread comprises computer readable code config- 
ured to cause one of said plurality of processors to 
select a thread from other processor's local dis- 
patch queue when there is no runnable thread in 
said processor's own local dispatch queue. 

17. The article of manufacture of claim 1 3 further com- 
prising: 

computer readable code configured to cause 
one of said plurality of processors to select one 
of said plurality of local dispatch queues to 
place a thread on; 

computer readable code configured to cause 
one of said plurality of processors to place the 
thread on said processor's local dispatch 
queue when the thread is bound. 

18. The article of manufacture of claim 1 3 further com- 
prising computer readable code configured to 
cause one of said plurality of processors to place 
the thread on said global queue when the thread 
has real time property. 



erty. 

20. The method of claim 1 3 wherein said step of notify- 
ing processors of the candidate thread comprising: 
computer readable code configured to cause 
one of said plurality of processors to change a mem- 
ory register value, said register accessible by said 
plurality of processors. 
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19. The article of manufacture of claim 1 8 further com- 
prising: 60 

computer readable code configured to cause 
one of said plurality of processors to identify a 
last processor that the thread ran on; 
computer readable code configured to cause ss 
one of said plurality of processors to place the 
thread on the last processor's dispatch queue 
when the thread does not have real time prop- 
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